Section: New Results

Brick & Mortar Cookies

Participants : Julien Badie, Manikandan Bakthavatchalam, Vasanth Bathrinarayanan, Ghada Balhoul, Anais Ducoffe.

The objective of the BMC project is to create a software that aims to present attendance and attractiveness of the customer in stores, based on automatic video analysis. The final system should be designed to be used without changing the current camera network of the customer store, dedicated to security purpose. Analysis should be given at different time and space resolutions. For instance, attendance of one particular day can be as interesting as attendance of the entire year. Moreover, shop owners want to be able to compare two given years or months, etc... As space resolution is concerned, the software should be able to give information about the global attractiveness of the store but should also analyze some specific zones.

IVA embedded on Bosch cameras

Intelligence Video Analysis (IVA) is embedded in some models of Bosch cameras. The algorithms are composed of human detection and tracking. They can be configured directly on the camera interface via tasks.

We are using a live connection to get metadata directly from the camera stream using a RTSP connection. Thie year we improved the results of last year using calibration tool embedded in the camera : shape of people detected was better, feet were followed with more precision as bounding boxes were more stable. We also tested the new IVA developed by BOSCH which was built to better manage changes in scene brightness and crossing of people. In the former version people close to each other were often detected as one person. Our first tests in shop revealed that it reduces the number of false detection but people were detected later than in the previous version. The case of people crossing doesn't seem to be better managed than before.

Inria algorithms : people detection and tracking

The previously enumerated tasks use algorithms to detect people and get their trajectories. Stars team has developed similar algorithms and has adapted their parameters values to the specific needs of this software. To improve results after some tests made during summer, the people detection is now using a deep learning method. People are detected earlier than before with this new algorithm and people crossing and occlusions are far better managed. The performances and the reliability of those algorithms were tested using an annotation tool developed in Stars Team.

Annotation tool

Manual annotation of videos requires major human effort. It can take hours and hours of fastidious work to annotate a tiny set of data. That's why we propose a semi-automatic tool which reduces the time of the annotation. This new semi automatic annotation tool uses a simple input data format, XML file or XGTF file to describe the video contents and algorithms output. Users only have to correct false or missing detection and to fix some wrong object id of the algorithms results using the annotation tool interface.

Tests in real conditions

We tested our video acquisition tool and our people detection and people tracking algorithms during summer in a partner supermarket in Nice. We successfully acquire 2 weeks of the desired metadata. By the end of summer, our results were highly improved by using a deep learning method to detect people. Moreover we can get results in quasi real-time. Except for the video stream acquisition tool, which needs to be connected to the camera network, our system is now running on an independent and local network. In case there is a crash of our system, the supermarket network will not be affected. Moreover, sensitive data are protected. A test is starting soon in SuperU to run and evaluate this new prototype.

Metadata storage in database

Last year metadata outputs of our analysis were first stored in XML files. Now to manage the quasi real-time solution, metadata are stored directly in the database we designed last year. We improve architecture of this database to manage simultaneously several connections as the final solution is supposed to be composed of several servers which will manage several video streams at the same time.

Web interface (HIM)

The web graphic interface is in progress. User interactions were added and improved so that the interface should be more user-friendly. We also changed some charts and tables so that statistical results should be better understood by users.